On models, criteria and search strategies for motion estimation in image sequences

Motion estimation is one of the key techniques helping solve problems encountered in image sequence compression and processing, and in computer vision. Redundancy elimination in digital video and tracking of moving objects in surveillance applications are but two interesting tasks where the knowledge of image motion is essential. Due to a strong correlation of image properties in the direction of motion, operations such as prediction, interpolation or ltering are most eecient when applied along motion trajectories. To compute these trajectories, underlying models need to be speciied, estimation criteria must be selected and a search strategy for model parameters must be implemented. In this paper, we expose each of those issues in a tutorial-like fashion. First, we discuss various motion representations and their relationship with the images. Then, we describe various estimation criteria: from a simple square of the displaced frame diierence to complex Bayesian criteria involving multiple terms. Finally, we address search strategies. We describe matching-and gradient-based schemes, deter-ministic and stochastic relaxation methods, including simulated annealing, as well as other deterministic approaches such as the \highest conndence rst" and mean eld techniques. We sketch multiresolution and multiscale strategies and point out their beneets. Throughout the paper we illustrate the theoretical discussion with experimental results. The paper is intended for readers involved in video compression, processing or computer vision, and in particular for those somewhat familiar with motion estimation looking to broaden their scope of knowledge.